Media overlay publication system

ABSTRACT

Systems and methods are presented that provide for receiving, at a media overlay publication system from a first client device, content to generate a media overlay, and generating the media overlay using the content received from the client device. The generated media overlay is stored in a database associated with the media overlay publication system and associated with a first characteristic of the content received from the first client device. The media overlay is provided to a second client device when a second characteristic of context data associated with the second client device correlates to the first characteristic for the media overlay, causing a display of the media overlay on a user interface of the second client device.

PRIORITY CLAIM

This application is a continuation of and claims the benefit of priorityof U.S. patent application Ser. No. 15/673,137, filed on Aug. 9, 2017,which is a continuation of and claims the benefit of priority of U.S.patent application Ser. No. 14/974,321, filed on Dec. 18, 2015, each ofwhich are hereby incorporated by reference herein in their entirety

TECHNICAL FIELD

The subject matter disclosed herein generally relates to providingcontext relevant media augmentation. Specifically, the presentdisclosure addresses systems and methods for a platform for publishingcontext relevant media augmentations, for presentation on userinterfaces.

BACKGROUND

The number of digital photographs and videos taken with mobile wirelessdevices is increasingly outnumbering photographs taken with dedicateddigital and film-based cameras. Thus, there are growing needs to improvethe experience associated with mobile wireless digital photography andvideography.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate exampleembodiments of the present disclosure and should not be considered aslimiting in scope.

FIG. 1 is a network diagram depicting a network system having aclient-server architecture configured for exchanging data over anetwork, according to one embodiment.

FIG. 2 shows a block diagram illustrating one example embodiment of amessaging application.

FIG. 3 shows a block diagram illustrating one example embodiment of amedia overlay application.

FIG. 4A shows a block diagram illustrating one example embodiment of auser-based media overlay publication module.

FIG. 4B shows an example of a graphical user interface for a user-basedmedia overlay publication module.

FIG. 4C shows an example of an operation of the graphical user interfaceof FIG. 4B.

FIG. 4D illustrates an example of a publication of a user-based mediaoverlay.

FIG. 5A shows a block diagram illustrating one example embodiment of amerchant-based media overlay publication module.

FIG. 5B illustrates an example of a common geolocation.

FIG. 5C illustrates an example of a graphical user interface for amerchant-based media overlay publication module.

FIG. 5D illustrates an example of a bid from a first merchant using thegraphical user interface of FIG. 5C.

FIG. 5E illustrates an example of a bid from a second merchant using thegraphical user interface of FIG. 5C.

FIG. 5F illustrates an example of an operation of a merchant-based mediaoverlay.

FIG. 6A shows a block diagram illustrating one example embodiment of apredefined media overlay module.

FIG. 6B shows a diagram illustrating an example of a media overlay withlive data content.

FIG. 6C shows a diagram illustrating an example of a media overlay withpromotional content.

FIG. 6D shows a diagram illustrating an example of a collectible mediaoverlay.

FIG. 6E shows a diagram illustrating an example of a media overlay withviral content.

FIG. 6F shows a diagram illustrating an example of a media overlay withaudio content.

FIG. 7 shows an interaction diagram illustrating one example embodimentof an operation of the user-based media overlay publication module.

FIG. 8 shows an interaction diagram illustrating an example embodimentof an operation of the merchant-based media overlay publication module.

FIG. 9 shows a flow diagram illustrating one example embodiment of anoperation of the user-based media overlay publication module.

FIG. 10 shows a flow diagram illustrating one example embodiment of anoperation of the merchant-based media overlay publication module.

FIG. 11 shows a flow diagram illustrating one example embodiment of anoperation of a live event module.

FIG. 12 shows a flow diagram illustrating one example embodiment of anoperation of a social network module.

FIG. 13 shows a flow diagram illustrating one example embodiment of anoperation of a promotion module.

FIG. 14 shows a flow diagram illustrating one example embodiment of anoperation of a collection module.

FIG. 15 shows a flow diagram illustrating one example embodiment of anoperation of a progressive use module.

FIG. 16 shows a flow diagram illustrating one example embodiment of anoperation of a viral use module.

FIG. 17 shows a flow diagram illustrating one example embodiment of anoperation of an actionable module.

FIG. 18 shows a flow diagram illustrating one example embodiment of amedia overlay publication module.

FIG. 19 is a block diagram illustrating an example of a softwarearchitecture that may be installed on a machine, according to someexample embodiments.

FIG. 20 illustrates a diagrammatic representation of a machine in theform of a computer system within which a set of instructions may beexecuted for causing the machine to perform any one or more of themethodologies discussed herein, according to an example embodiment.

FIG. 21 is a block diagram illustrating a mobile device, according to anexample embodiment.

DETAILED DESCRIPTION

Although the present disclosure is described with reference to specificexample embodiments, it will be evident that various modifications andchanges may be made to these embodiments without departing from thebroader spirit and scope of the disclosure. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

The addition of labels, drawings, and other artwork (e.g., mediaoverlays) to images (e.g., photographs or video) provides a compellingway for users to personalize, supplement, and enhance these imagesbefore storage or publication to a broader audience. An exampleembodiment seeks to provide users with a set of the media overlays (orother enhancements and augmentations) that may be applied to an image.The set of enhancements and augmentations, in the example form of mediaoverlays, may be determined based on a location associated with theimage and other contextual information, such as ambient noise orenvironmental acoustic (audio) sound, particular objects associated withthe image, etc. The media overlays may be presented to a user forselection and combining with an image based on a determined location ofthe image, or content of the image. For example, where a user takes aphotograph or video on a mobile device at a music concert, a mediaoverlay indicating the name of the concert event and a band playing at acurrent time may be presented to the user for selection and overlay onthe photograph or video. In another example, where a user initiatestaking (or has taken) a photograph or video at Disneyland, an imageoverlay indicating the name “Disneyland”, in a particular style, ispresented to the user. Further Disneyland-themed image overlays may alsobe presented to the user. The presentation of the image overlay may bein response to the user performing a gesture (e.g., a swipe operation, along press) on a screen of the mobile device. The user is then able toselect the image overlay and have it applied to an image (e.g., an imagethat the user takes before the selection, or takes after the selection),in this way to personalize and enhance the image.

In an example embodiment, a media overlay may be presented to a userautomatically upon detection of a particular event. For example, when auser initiates taking (or has taken) a photograph or video, content inthe photograph or video (e.g., audio, an object, a location, etc.) cantrigger one or more media overlays to be displayed to the user forselection. For example, a user may be taking a video that includesaudio. A particular audio track may be identified from the audio and amedia overlay may be created that includes information such as the audiotrack name, title, artist name, album art, etc. (as examples). Thismedia overlay may be presented to the user to be incorporated with thevideo content or other content that the user has taken.

Third party entities (e.g., merchants, restaurants, individuals, etc.)may, in one example embodiment, seek to have overlays included in theset presented for user selection at a particular geographic location.For example, a restaurant at a particular location in San Francisco maywish to have its restaurant name and logo included in a set of mediaoverlays presented to a user, for the purposes of augmenting aphotograph taken by the user proximate to the restaurant. According toone example embodiment, such third party entities may bid (or otherwisepurchase opportunities) to have a particular media overlay included in aset presented to a user for augmentation of a particular image. Belowdescribed are various systems and methodologies that may be used totechnically implement the above described image enhancement technologiesand capabilities.

More specifically, various examples of a media overlay publicationapplication are described. The media overlay publication applicationoperates at a server and generates media overlays that include contentbased on geographic locations (also referred to as geolocations) andother contextual information (e.g., characteristics of a content item).Other media enhancements or augmentations may include audio and visualcontent or visual effects that may be applied to augment a content ormedia item (e.g., photograph or video) at a mobile device. The mediaoverlay publication application includes a user-based media overlaypublication platform and a merchant-based media overlay publicationplatform.

In the user-based media overlay publication platform, the media overlaypublication application may provide a Graphical User Interface (GUI) fora user to upload content and select a geolocation on a map. For example,the user may upload a logo and define boundaries on the map to identifya particular geolocation associated with the logo. Once the user submitsthe logo and identifies the particular geolocation, the media overlaypublication application may generate a media overlay that includes thelogo associated with the particular geolocation. As such, mobile devicesthat are located within the particular geolocation have access to themedia overlay.

In the merchant-based media overlay publication platform, the mediaoverlay publication application may provide a GUI for merchants toupload content, select geolocations on a map, and submit bids for thecorresponding geolocations. A bidding process may determine the merchantwith the highest bid amount. That merchant may then exclude publicationof media overlays from other merchants at a selected geolocation of themerchant. Therefore, the media overlay of the highest-bidding merchantmay be the only media overlay that can be accessed by mobile devicesthat are located at the selected geolocation.

In other examples, the media overlay includes context relevant data,such as a current temperature, an identification of a geolocation of themobile device (e.g., Venice Beach), a name of a live event associatedwith the geolocation of the mobile device, or a name of a business.

In some embodiments, a media overlay application at a server provides alive event media overlay to a mobile device. The live event mediaoverlay includes live event data associated with a live event, such as aconcert event, a sporting event, or an award ceremony, at a geolocationof the mobile device. For example, a user attending a football game canaccess a sports media overlay that includes the current score of thefootball game. In another example, a user attending the Oscars awardceremony can access an entertainment media overlay that includes a nameof an Oscar winner.

In other embodiments, the media overlay application at the serverprovides a social network media overlay to the mobile device. The socialnetwork media overlay may be based on social network activities of theuser of the mobile device. For example, if the user follows a brand suchas McDonald's on a social network service, and the mobile device of theuser is located at a McDonald's restaurant, the mobile device of theuser can access a McDonald's media overlay. Other users located at thesame restaurant would not have access to the McDonald's media overlayunless they also follow McDonald's on the social network service. Inanother example, the order in which the media overlays are presented tousers located at a McDonald's restaurant may be modified so that theMcDonald's media overlay is served higher for users following McDonald'son the social network service.

In further embodiments, the media overlay application at the serverprovides a promotion media overlay to mobile device. The promotion mediaoverlay may be based on promotions from a merchant. For example, themedia overlay may be used to implement a Monopoly™ game at McDonald's byrandomly selecting a media overlay every time the user of the mobiledevice walks into a McDonald's restaurant and purchases an item. Themedia overlay can be used to obtain Monopoly puzzle pieces that can beredeemed towards prizes.

In some embodiments, the media overlay application at the server enablesthe mobile device to collect media overlays. For example, the mobileoverlay application provides the mobile device with permanent access tocollected media overlays. The collected media overlays may be stored ina collection portfolio for the mobile device. The mobile device mayaccess any of the media overlays in the collection portfolio at anytime.

In some embodiments, the media overlay application at the serverprovides a history media overlay to the mobile device. The history mediaoverlay may be based on geographic locations of historical sites visitedby the user of the mobile device. For example, the mobile device isawarded with a unique media overlay associated with one of the SevenWonders of the World when the mobile device is located at one of thecorresponding Seven Wonders geographic locations.

In another embodiment, the media overlay application at the serverprovides a progressive use media overlay to the mobile device. Thecontent in the progressive use media overlay changes depending on thenumber of people that have previously used the progressive use mediaoverlay.

In another example embodiment, users can “purchase” a geolocation for apredetermined amount of time and select a media overlay associated withthe geolocation. For example, a college can purchase and select aparticular media overlay associated with the geolocation of its campus.

In another example embodiment, the media overlay application provides aviral media overlay to the mobile device. For example, when the user ofthe mobile device obtains the viral media overlay at a geolocation, thatuser can send the viral media overlay to mobile devices located outsidethe geolocation of the original user. Users of the mobile deviceslocated outside the geolocation of the original user can make use of theviral media overlay for the next hour. Those users can also forward theviral media overlay to other users.

In another example embodiment, the media overlay application provides anactionable media overlay to the mobile device. For example, theactionable media overlay can be a link to open a browser page in themobile device to obtain a coupon. The actionable media overlay cantrigger other functions of the mobile device.

FIG. 1 is a network diagram depicting a network system 100 having aclient-server architecture configured for exchanging data over anetwork, according to one embodiment. For example, the network system100 may be a messaging system where clients may communicate and exchangedata within the network system 100. The data may pertain to variousfunctions (e.g., sending and receiving text and media communication,determining geolocation) and aspects (e.g., publication of mediaoverlays, management of media overlays) associated with the networksystem 100 and its users. Although the network system 100 is illustratedherein as having a client-server architecture, other embodiments mayinclude other network architectures, such as peer-to-peer or distributednetwork architectures.

A data exchange platform 102, in an example, includes a messagingapplication 120 and a media overlay application 122, and may provideserver-side functionality via a network 104 (e.g., the Internet) to oneor more clients. The one or more clients may include users that utilizethe network system 100 and, more specifically, the messaging application120 and the media overlay application 122, to exchange data over thenetwork 104. These operations may include transmitting, receiving(communicating), and processing data to, from, and regarding content andusers of the network system 100. The data may include, but is notlimited to, content and user data such as user profiles, messagingcontent, messaging attributes, media attributes, client deviceinformation, geolocation information, media overlay content, messagingcontent persistence conditions, social network information, and liveevent data information, among others.

In various embodiments, the data exchanges within the network system 100may be dependent upon user-selected functions available through one ormore client or user interfaces (UIs). The UIs may be associated with aclient machine, such as client devices 110, 112 using a programmaticclient 106, such as a client application. The programmatic client 106may be in communication with the messaging application 120 and mediaoverlay application 122 via an application server 118. The clientdevices 110, 112 include mobile devices with wireless communicationcomponents, and audio and optical components for capturing various formsof media including photos and videos. The client devices 110 and 112 mayfurther include a global positioning system (GPS) processor 107.

Turning specifically to the messaging application 120 and the mediaoverlay application 122, an application program interface (API) server114 is coupled to, and provides a programmatic interface to, one or moreapplication server(s) 118. The application server 118 hosts themessaging application 120 and the media overlay application 122. Theapplication server 118 is coupled to one or more database servers 124that facilitate access to one or more databases 126.

The API server 114 communicates and receives data pertaining to messagesand media overlays, among other things, via various user input tools.For example, the API server 114 may send and receive data to and from anapplication (e.g., the programmatic client 106) running on anotherclient machine (e.g., client devices 110, 112 or a third party server).

In one example embodiment, the messaging application 120 providesmessaging mechanisms for users of the client devices 110, 112 to sendmessages that include text and media content such as pictures and video.The client devices 110, 112 can access and view the messages from themessaging application 120 for a limited period of time. For example, theclient device 110 can send a message to the client device 112 via themessaging application 120. Once the client device 112 accesses themessage from the messaging application 120, the message is deleted aftera predefined duration has elapsed from the time the client device 112started viewing the message. Components of the messaging application 120are described in more detail below with respect to FIG. 2.

In one example embodiment, the media overlay application 122 provides asystem and a method for operating and publishing media overlays formessages processed by the messaging application 120. The media overlayapplication 122 supplies a media overlay to the client device 110 basedon characteristics of a content item (e.g., photograph or video) or ageolocation of the client device 110. In another example, the mediaoverlay application 122 supplies a media overlay to the client device110 based on other information, such as social network information ofthe user of the client device 110.

The media overlay may include audio and visual content and visualeffects. Examples of audio and visual content include pictures, texts,logos, animations, and sound effects. An example of a visual effectincludes color overlaying. The audio and visual content or the visualeffects can be applied to a media content item (e.g., a photo) at theclient device 110. For example, the media overlay includes text that canbe overlaid on top of a photograph generated at the client device 110.In another example, the media overlay includes an identification of alocation overlay (e.g., Venice Beach), a name of a live event, a name ofa merchant overlay (e.g., Beach Coffee House), or a name of a song. Inanother example, the media overlay application 122 uses the geolocationof the client device 110 to identify a media overlay that includes thename of a merchant at the geolocation of the client device 110. Themedia overlay may include other indicia associated with the merchant.Examples of indicia include logos and other pictures related to themerchant. The media overlays may be stored in the database(s) 126 andaccessed through the database server 124.

In one example embodiment, the media overlay application 122 includes auser-based publication platform that enables users to select ageolocation on a map, and upload content associated with the selectedgeolocation. The user may also indicate other circumstances under whicha particular media overlay can be provided. The media overlayapplication 122 generates a media overlay that includes the uploadedcontent and associates the uploaded content with the selectedgeolocation.

In another example embodiment, the media overlay application 122includes a merchant-based publication platform that enables merchants toselect a particular media overlay associated with a geolocation via abidding process. For example, the media overlay application 122associates the media overlay of a highest-bidding merchant with acorresponding geolocation for a predefined amount of time. Components ofthe media overlay application 122 are described in more detail belowwith respect to FIG. 3.

FIG. 2 shows a block diagram illustrating one example embodiment of themessaging application 120. The messaging application 120 may be hostedon dedicated or shared server machines (not shown) that arecommunicatively coupled to enable communications between the servermachines. The messaging application 120 and the media overlayapplication 122 themselves are communicatively coupled (e.g., viaappropriate interfaces) to each other and to various data sources, so asto allow information to be passed between the messaging application 120and the media overlay application 122, or so as to allow the messagingapplication 120 and the media overlay application 122 to share andaccess common data. The messaging application 120 and the media overlayapplication 122 may, furthermore, access the one or more databases 126via the database server(s) 124.

The messaging application 120 is responsible for the generation anddelivery of messages between users of the programmatic client 106. Themessaging application 120 may utilize any one of a number of messagedelivery networks and platforms to deliver messages to users. Forexample, the messaging application 120 may deliver messages usingelectronic mail (e-mail), instant message (IM), Short Message Service(SMS), text, facsimile, or voice (e.g., Voice over IP (VoIP)) messagesvia wired (e.g., the Internet), plain old telephone service (POTS), orwireless networks (e.g., mobile, cellular, WiFi, Long Term Evolution(LTE), Bluetooth).

In one example embodiment, the messaging application 120 includes amedia receiver module 202, a media overlay application interface 204, amessage generator module 206, an ephemeral message access module 208,and an ephemeral message storage module 210. The media receiver module202 receives a message from the programmatic client 106 of the clientdevice 110. The message may include a combination of text, photo, orvideo. The media receiver module 202 also receives persistence metadataassociated with the message. The persistence metadata defines how long amessage can be viewed. For example, the user of the client device 110may specify that the message be persistent or can only be viewed oraccessed for a user-determined amount of time (e.g., ten seconds). Themedia overlay application interface 204 communicates with the mediaoverlay application 122 to access and retrieve a media overlayassociated with the metadata in the message. The message generatormodule 206 applies the media overlay to the message from theprogrammatic client 106 to create an ephemeral message, and temporarilystores the ephemeral message using the ephemeral message storage module210.

The ephemeral message access module 208 notifies a recipient of themessage of the availability of the ephemeral message. The ephemeralmessage access module 208 receives a request to access the ephemeralmessage from the recipient and causes the ephemeral message to bedisplayed on a client device of the recipient for the maximum durationspecified in the persistence metadata. Once the recipient views themessage for the maximum duration, the ephemeral message access module208 causes the client device of the recipient to stop displaying theephemeral message, and deletes the ephemeral message from the ephemeralmessage storage module 210.

FIG. 3 shows a block diagram illustrating one example embodiment of themedia overlay application 122. The media overlay application 122includes a media overlay publication module 304 and a media overlayengine 306.

The media overlay publication module 304 provides a platform forpublication of media overlays. In an example embodiment, the mediaoverlay publication module 304 includes a user-based media overlaypublication module 314 and a merchant-based media overlay publicationmodule 316. The user-based media overlay publication module 314 enablesusers of client devices (either mobile or web clients) to upload contentand select a geolocation for a user-based media overlay. Themerchant-based media overlay publication module 316 enables merchants toupload content, select a geolocation, and submit a bid amount for amerchant-based media overlay. The user-based media overlay publicationmodule 314 is described in more detail below with respect to FIG. 4A.The merchant-based media overlay publication module 316 is described inmore detail below with respect to FIG. 5A.

The media overlay engine 306 generates and supplies a media overlaybased on a characteristic of a content item or a geolocation of a clientdevice. In one example embodiment, the media overlay engine 306 includesa predefined media overlay module 318, a user-based media overlay module320, and a merchant-based media overlay module 322. The media overlaymay be based on predefined media overlays from the predefined mediaoverlay module 318, user-based media overlays from the user-based mediaoverlay module 320, and merchant-based media overlays from themerchant-based media overlay module 322.

The predefined media overlay module 318 supplies the client device withone of a set of predefined media overlays. Examples of predefined mediaoverlays are described in more detail below with respect to FIG. 6A.

The user-based media overlay module 320 supplies the client device witha user-based media overlay generated by the user-based media overlaypublication module 314. The merchant-based media overlay module 322supplies the client device with a merchant-based media overlay generatedby the merchant-based media overlay publication module 316.

FIG. 4A shows a block diagram illustrating one example embodiment of theuser-based media overlay publication module 314. The user-based mediaoverlay publication module 314 includes a user-based content uploadmodule 402, a user-based geolocation selection module 404, a user-basedduration selection module 406, and a user-based publication engine 408.

The user-based content upload module 402 receives uploaded content froma user. The content may include a media item such as a photo or a video.The user-based content upload module 402 may be implemented on a webserver to allow a user to upload the content using a GUI as illustratedin FIG. 4B.

The user-based geolocation selection module 404 receives geolocationidentification information from the user to identify a selectedgeolocation. The geolocation identification information may include anaddress, an identification of an establishment already associated withthe address, Global Positioning System (GPS) coordinates, or ageographic boundary. For example, the address may include a streetnumber, street address, city, state, and country. The user may alsoidentify a location based on an existing establishment. For example, thegeolocation information may include “restaurant x” in Venice Beach. Thegeographic boundary identifies a region or a zone. For example, thegeographic boundary may define a region located within a predeterminedradius of an address, a point of interest, or an existing establishment.

In one example embodiment, the geolocation identification informationmay be embedded in a message or communication from a client device tothe user-based geolocation selection module 404. For example, the userof the client device may take a picture of a sunset at Venice Beach andsend the picture to the user-based geolocation selection module 404,which may then extract the geolocation attribute or identificationinformation from metadata associated with the picture of the sunset. Theuser-based geolocation selection module 404 may be implemented on a webserver to present a user with a GUI in a web page that allows the userto select the geolocation for the content as illustrated in FIG. 4C.

The user-based duration selection module 406 receives, from the user,time duration information related to the uploaded content and selectedgeolocation. The time duration information may identify a period of timeduring which the uploaded content is associated with the selectedgeolocation. Once the period of time has elapsed, the uploaded contentis no longer associated with the selected geolocation. For example, ifthe time duration indicates twenty-four hours, the media overlay engine306 makes the user-based media overlay available to client devices thatare located at the selected geolocation. Once twenty-four hours haselapsed, the user-based media overlay is no longer accessible by theclient devices at the selected geolocation.

Other embodiments include periodic time duration information or specifictime duration information. For example, for the periodic time durationinformation, the user-based media overlay is published and madeavailable at the selected geolocation every Sunday (e.g., areligion-related media overlay available on days of religious services).For the specific time duration information, the user-based media overlayis published and made available at the selected geolocation around aspecific holiday or date (e.g., Thanksgiving weekend, New Year's day).

The user-based publication engine 408 generates a user-based mediaoverlay that associates the uploaded content from the user-based contentupload module 402 with the selected geolocation from the user-basedgeolocation selection module 404. The user-based publication engine 408publishes the user-based media overlay to client devices that arelocated within the selected geolocation for the time duration identifiedwith the user-based duration selection module 406.

In another example embodiment, the user-based publication engine 408determines that no other user-based media overlays exist during the sameperiod of time for the same selected geolocation. The user-basedpublication engine 408 may publish just one user-based media overlay atany time for the same selected geolocation. In another exampleembodiment, a limit may be placed on the number of user-based mediaoverlays available at any time for the same selected geolocation. Thus,the user-based publication engine 408 may publish and make available alimited number of user-based media overlays at any time for the sameselected geolocation. In another example embodiment, user-based mediaoverlays may be published to only contacts or “friends” of the uploadinguser.

FIG. 4B illustrates an example of a GUI 410 for uploading content andfor selecting a geographic region on a map. The GUI 410 includes a map412, an upload image box 414, a select location button 416, a overlaytitle box 418, and a submit button 420. The upload image box 414 enablesa user to upload content (e.g., a picture) to the user-based contentupload module 402. The select location button 416 enables the user toidentify a geolocation by drawing boundaries on the map 412 or byinputting an address or a zip code. The identified geolocation issubmitted to the user-based geolocation selection module 404. Theoverlay title box 418 enables the user to submit a name for the mediaoverlay. The user may submit the content and the requested geolocationby clicking on the submit button 420. Once the content and requestedgeolocation are submitted, the user-based publication engine 408generates a user-based media overlay that includes the uploaded contentfor the identified geolocation.

FIG. 4C illustrates an example where user-identified boundary points424, 426, 428, and 430 on the map 412 define a geolocation 422. The userhas uploaded a picture of the sun 415 displayed in the upload image box414. The user has entered the title of the content “Fun in the sun!” inthe overlay title box 418. The user may submit the picture of the sun415 and the geolocation 422 by clicking on the submit button 420. Oncethe picture of the sun 415 and the geolocation 422 are submitted, theuser-based publication engine 408 generates a user-based media overlay.

FIG. 4D illustrates an example of a publication of a user-based mediaoverlay. The media overlay application 122 detects that a mobile device1802 of a user 1816 is located at the geolocation 422. The media overlayapplication 122 retrieves a user-based media overlay 440 correspondingto the geolocation 422 and publishes the user-based media overlay 440 tothe mobile device 1802. The user-based media overlay 440 is applied tomedia content 1806 in a display 1804 of the mobile device 1802.

FIG. 5A shows a block diagram illustrating one example embodiment of themerchant-based media overlay publication module 316. The merchant-basedmedia overlay publication module 316 includes a merchant-based contentupload module 502, a merchant-based geolocation selection module 504, amerchant-based duration selection module 506, a merchant-based biddingmodule 508, and a merchant-based publication engine 510.

The merchant-based content upload module 502 receives content from amerchant. The content may include a media item such as a picture, avideo, a graphic, or a text. The merchant-based content upload module502 may be implemented on a web server to allow a merchant to upload thecontent using a webpage.

The merchant-based geolocation selection module 504 receives geolocationidentification information from the merchant to identify a selectedgeolocation. The geolocation identification information may include anaddress of an establishment, an identification of an establishmentalready associated with the address, GPS coordinates, or a geographicboundary. For example, the address of the establishment may include astreet number, street address, city, state, and country. The merchantmay also identify a location based on an existing establishment. Forexample, the geolocation information may include “restaurant x” inVenice Beach. The geographic boundary identifies a region or a zone. Forexample, the geographic boundary may define a region located within apredetermined radius of an address, a point of interest, or an existingestablishment. The merchant may further define the geographic boundaryby drawing a virtual fence on a map. The merchant-based geolocationselection module 504 may be implemented on a web server to allow amerchant to draw boundaries on a map in a web page.

The merchant-based duration selection module 506 receives, from themerchant, time duration information related to the uploaded content andselected geolocation. The time duration may identify a period of time inwhich the uploaded content is associated with the selected geolocation.Once the period of time has elapsed, the uploaded content is no longerassociated with the selected geolocation. Other embodiments includeperiodic time duration information or specific time durationinformation. For example, for the periodic time duration information,the merchant-based media overlay is published or made available at theselected geolocation (e.g., corner of two identified streets) everySaturday night (e.g., a nightclub-related media overlay available everySaturday night). For the specific time duration information, theselected media overlay is published or made available at the selectedgeolocation around a specific date (e.g., party event date).

The merchant-based bidding module 508 provides an interface to enablemerchants to submit a bid amount for a common geolocation. The commongeolocation may include, for example, a same street address. Forexample, several businesses may have the same street address butdifferent suite numbers in a shopping center. FIG. 5B illustrates anexample of a common geolocation 516. Merchant A geolocation boundaries512 overlap with merchant B geolocation boundaries 514 to define acommon geolocation 516. Thus, merchants A and B may submit respectivebids corresponding to the common geolocation 516. In one exampleembodiment, the merchant-based geolocation selection module 504determines common geolocations from the geolocations selected by themerchants. The merchant-based bidding module 508 identifies a highestbidder for the common geolocation 516 and awards the highest bidder withthe ability to exclude other merchant-based media overlays from thecommon geolocation 516 for a predefined amount of time.

In another example embodiment, the merchant-based bidding module 508prorates bid amounts based on their corresponding time durationinformation. For example, merchant A submits a bid amount of $100 forone day for a specific geolocation. Merchant B submits a bid amount of$160 for two days for the same specific geolocation. The merchant-basedbidding module 508 may prorate the bid from merchant B for one day(e.g., $80) and compare both bids for the same period of time (e.g., oneday) to determine a highest bidder.

The merchant-based publication engine 510 generates a merchant-basedmedia overlay that associates the uploaded content of the highest bidderwith the geolocation identified by the highest bidder. Themerchant-based publication engine 510 publishes the merchant-based mediaoverlay to client devices that are located at the geolocation selectedby the highest bidder for the time duration identified with themerchant-based duration selection module 506. Merchant-based mediaoverlays from other merchants in the common geolocation 516 are excludedfrom publication. In another embodiment, a quota may be placed on thenumber of merchant-based media overlays available for the commongeolocation 516. For example, the merchant-based publication engine 510may publish and make available a limited number of merchant-based mediaoverlays (e.g., a maximum of two merchant-based media overlays) for thecommon geolocation 516.

In another example embodiment, the merchant-based publication engine 510forms a priority relationship that associates the uploaded content ofthe highest bidder with the geolocation selected by the highest bidder.For example, an order in which media overlays are displayed at theclient device 110 may be manipulated based on the results from themerchant-based bidding module 508. A media overlay of a merchant withthe highest bid may be prioritized and displayed first at the clientdevice 110. Media overlays from other merchants may be displayed at theclient device 110 after the media overlay of the highest bidder. Inanother example embodiment, a merchant may be able to bid on alllocations at which it maintains a presence. Thus, a restaurant chain maybe able to have its media overlay(s) published at each of its restaurantchain locations.

FIG. 5C illustrates an example of a GUI 520 for uploading content andfor selecting a geolocation on a map. The GUI 520 includes a map 522, anupload image box 524, a select location button 526, an overlay title box528, a bid amount entry box 530, a campaign length entry box 532, and asubmit button 534. The upload image box 524 enables a merchant to uploadcontent (e.g., a picture, a video, or an animation) to themerchant-based content upload module 502. The select location button 526enables the merchant to identify a geolocation by drawing boundaries onthe map 522 or by inputting an address or a zip code. The overlay titlebox 528 enables the merchant to submit a name for the media overlay. Thebid amount entry box 530 enables the merchant to enter a bid amount forthe identified geolocation. The campaign length entry box 532 enablesthe merchant to specify a length of a campaign in which the uploadedcontent is associated with the identified geolocation. The merchant maysubmit the uploaded content and entered information by clicking on thesubmit button 534.

FIG. 5D illustrates an example where a merchant A has identifiedboundary points 542, 544, 546, and 548 on the map 522 to define ageolocation 540. Merchant A has uploaded a picture 525 displayed in theupload image box 524. Merchant A has entered a title “Coffee shop A” inthe overlay title box 528, a bid amount of $300 in the bid amount entrybox 530, and a campaign length of 30 days in the campaign length entrybox 532. Merchant A submits the picture 525, the requested geolocation540, and other entered information by clicking on the submit button 534.The merchant-based publication engine 510 generates a media overlay formerchant A.

FIG. 5E illustrates an example where another merchant, merchant B, hasidentified boundary points 552, 554, 556, and 558 on the map 522 todefine a geolocation 550. Merchant B has uploaded a picture 527displayed in the content upload box 524. Merchant B has entered a title“Coffee shop B” in the overlay title box 528, a bid amount of $500 inthe bid amount entry box 530, and a campaign length of 30 days in thecampaign length entry box 532. Merchant B may submit the picture 527,the requested geolocation 550, the bid amount, and the campaign lengthby clicking on the submit button 534. The merchant-based publicationengine 510 generates a media overlay for merchant B.

FIG. 5F shows a diagram illustrating an example of a merchant-basedmedia overlay selected based on a bidding process. The geolocation 540of merchant A and the geolocation 550 of merchant B overlap at a commongeolocation 545. The user 1816 is located at the common geolocation 545and uses the mobile device 1802 to generate the media content 1806(e.g., user 1816 takes a picture) in the display 1804 of the mobiledevice 1802. The media overlay of the merchant with the highest bid forthe common location 545 is published to the mobile device 1802. In thepresent example, merchant B has outbid merchant A. As such, a mediaoverlay 560 of merchant B is provided and displayed in the display 1804on top of the media content 1806. The media overlay 560 contains theuploaded content from merchant B. In addition, “merchant” in the contextof the current example embodiments may include not only entitiesinvolved in the trade or sale of merchandise but any other entity aswell, including individuals, universities, non-profit organizations,student organizations, clubs, etc.

FIG. 6A shows a block diagram illustrating one example embodiment of thepredefined media overlay module 318. The predefined media overlay module318 includes, for example, a live event module 602, a social networkmodule 604, a promotion module 606, a collection module 608, aprogressive use module 610, a viral use module 612, an actionable module614, a history aware module 616, and an audio module 618.

The live event module 602 generates a media overlay based on live eventinformation. The live event information may be related to a live gamescore of a sporting event associated with a corresponding geolocation,or a live news event related to an entertainment (e.g., concert) orsocial event associated with a corresponding geolocation. For example, auser of the client device 110 attends a live sporting event (e.g., agame) at a stadium. As such, media metadata from the client device 110may identify the location of the stadium with a date and time. The liveevent module 602 uses that information to search for a live eventassociated with the location of the stadium, date, and time. The liveevent module 602 retrieves a current or nearly current game scoreassociated with the live sporting event at the stadium (e.g., via theESPN API). The live event module 602 may also retrieve insignias or teamlogos associated with the live sporting event. As such, the live eventmodule 602 generates a media overlay containing the latest score basedon news sources covering the live sporting event.

In another example, the user of the client device 110 attends a livesocial event at a venue. Similarly, media metadata identifies thelocation of the venue with a date and time. The live event module 602uses that information to search for a live event associated with thelocation of the venue, date, and time from sources such as a socialnetwork server or news media service. The live event module 602retrieves a news feed associated with the live social event at thevenue. As such, the live event module 602 generates a media overlaycontaining information or content based on news retrieved from a newsfeed associated with the live social event at the venue.

The social network module 604 generates a media overlay based on socialnetwork information of a user of the client device 110. The socialnetwork information may include social network data retrieved from asocial network service provider. The social network data may includeprofile data of the user, “likes” of the user, establishments that theuser follows, friends of the user, and postings of the user, amongothers. For example, the media overlay associated with a restaurant maybe available to the user at the location of the restaurant if the userhas identified himself as a fan of the restaurant or indicates a “like”of the restaurant with the social network service provider. In anotherexample, the ranking or priority of displaying the media overlay in theclient device 110 of the user may be based on the profile of the user orthe number of “check-ins” of the user at the restaurant.

In another example embodiment, the media overlay may be restricted andavailable only to the user and the social network (e.g., friends orother users in different categories) of the user of the client device110. As such, the user may forward the media overlay to his friends.

The promotion module 606 generates media overlays for a promotion (e.g.,a game, contest, lottery). For example, a set of unique media overlaysmay be generated. One media overlay from the set of unique mediaoverlays may be provided to the client device 110 when the client device110 is at a predefined location associated with the media overlays orwhen an object (e.g., name, logo, product, etc.) is recognized in aphotograph or video taken by the user. For example, the user may visit afast food restaurant. The media metadata from the client device 110identifies the location of the fast food restaurant. The promotionmodule 606 retrieves a unique media overlay from the set of unique mediaoverlays and provides it to the client device 110. The promotion module606 may remove the unique media overlay from the set of unique mediaoverlays after it has been provided to the client device 110. In anotherembodiment, the promotion module 606 removes the unique media overlayfrom the set of unique media overlays after it has been provided toother client devices for a predefined number of times.

In another example, a promotional media overlay may be generated basedon occurrences in another application (e.g., communicated to the mediaoverlay application 122 via an API). For example, the media overlay maybe an image of a character from a video game provided to the clientdevice 110, or made available to a user, when the user completes a levelin the video game. The media overlay may be available for a single use,multiple uses, for a period of time, or indefinitely.

In another example, a media overlay may be provided to the client device110 or made available to a user based on another account that the userhas linked with the media overlay application (e.g., via an accountassociated with the media overlay application). For example, the usermay have an Xbox account that is linked to the user's account for themedia overlay application. The user may be a first user and the firstuser may beat a second user on Xbox (e.g., playing a football game). Thegame or application on the Xbox may communicate the fact that the firstuser beat the second user on Xbox to the media overlay application 122(e.g., via an API). A media overlay may be generated and provided to theclient device 110 or made available to the first user, that includesinformation regarding the first user beating the second user. Forexample, the media overlay may include the fact that the first user beatthe second user, the score, the overall record of the first user vs. thesecond user, etc.

In another example, a user may have linked a frequent flier account(e.g., an American Airlines account) to the media overlay application.When the user is boarding an airplane for a flight or is on a flightassociated with the frequent flier account (e.g., on American Airlinesflight number 10 from LAX to JFK), the user may receive a media overlayassociated with the flight. For example, the user may receive a mediaoverlay that includes the airlines logo, the flight route, the departureand arrival time, a decorative image of the departure and arrival cities(e.g., Los Angeles and New York), etc.

The media overlay includes content related to a game or promotion. Inanother example, the media overlay may include dynamic content adjustedbased on the game or promotion. For example, the dynamic content mayinclude a current number of remaining media overlays of the game orpromotion. The media overlays from the promotion module 606 may be“collected” by the client device 110. For example, the client device 110may store the media overlay in a collection at the client device 110. Aprize may be redeemed upon collection of each overlay of a predefinedset of media overlays.

The collection module 608 generates collectible media overlays. Forexample, the client device 110 is provided with a media overlayassociated with the geolocation of the client device 110. The mediaoverlay may be collected by the client device 110 and be madepermanently available to the client device 110. The client device 110may store the collected media overlay in a collection folder at theclient device 110.

The progressive use module 610 generates media overlays with dynamiccontent that changes based on a number of uses of the media overlays.For example, a media overlay can be set to be used for a limited numberof times. Every time the media overlay is provided to a client device, acontent of the media overlay is adjusted. For example, the media overlaymay include a fundraising progress bar in which a level of the bar risesevery time the media overlay is used. The dynamic content in the mediaoverlay may include a countdown displaying the number of remainingusages of the media overlay.

The viral use module 612 generates media overlays that can be forwardedto other users outside a geolocation associated with the media overlays.For example, the client device 110 receives a media overlay based on ageolocation of the client device 110. The client device 110 can send themedia overlay to the client device 112 that is outside the geolocationof the client device 110. The forwarded media overlay may be availablefor use by the client device 112 for a predefined time limit (e.g., onehour). Similarly, the client device 112 may forward the media overlay toother client devices outside the geolocation of the client device 110for use within the predefined time limit.

The actionable module 614 generates media overlays with an actionassociated with a content item of the media overlays. For example, themedia overlay can start a browser of the client device 110 and open apredetermined website in the browser. In another embodiment, the mediaoverlay is capable of opening other functionalities (e.g., a paymentapplication) or executing other programs at the client device 110. Forexample, a user can tap on the media overlay to download or display acoupon associated with the media overlay at the client device 110.

The history aware module 616 generates media overlays based ongeolocation of the client device 110 and historical events associatedwith the geolocation. For example, a media overlay may include picturesof a pyramid associated with the geolocation of the client device 110.The media overlays may be collected based on the historical events or,for example, for each of the Seven Natural Wonders of the World. Forexample, a media overlay associated with a national park may becollected when the user visits the national park. The client device 110can collect all media overlays associated with all national parks.

The audio module 618 generates media overlays based on audio content(e.g., an audio track) associated with a content item received from aclient device 110. For example, a media overlay may include one or moreof a title associated with the audio content, an artist name associatedwith the audio content, an album name associated with the audio content,album artwork associated with the audio content, lyrics associated withthe audio content, etc. In another embodiment, the media overlay iscapable of opening other functionalities (e.g., to buy the songassociated with the audio content, to get more information about thesong, etc.). For example, a user can tap on the media overlay todownload or display a coupon associated with the media overlay at theclient device 110.

FIG. 6B shows a diagram illustrating an example of a media overlay 1820with live data content. The media overlay 1820 contains live dataassociated with a geolocation of the mobile device 1802. For example,the live data contains a live weather status 1822 and a latest scoreupdate 1824 of a sporting event associated with the geolocation of themobile device 1802. The mobile device 1802 displays the media overlay1820 on top of (e.g., as a transparent overlay) the media content 1806on the display 1804. In one example embodiment, the media overlay 1820may be implemented with the live event module 602 of FIG. 6A.

FIG. 6C shows a diagram illustrating an example of a media overlay 1830with promotional content. For example, the media overlay 1830 includes adigital coupon 1832 that can be redeemed at a coffee shop. The mediaoverlay 1830 may include dynamic content 1834. For example, the dynamiccontent 1834 may include a remaining number of times the coupon can beused. Furthermore, the media overlay 1830 may include an actionable area1836 that is associated with an executable function. For example, whenthe user taps the actionable area 1836, the media overlay 1830 isforwarded to a mobile device of a friend of the user. The mobile device1802 displays the media overlay 1830 on top of the media content 1806 onthe display 1804. In one example embodiment, the media overlay 1830 maybe implemented with the social network module 604, the promotion module606, the progressive use module 610, and the actionable module 614 ofFIG. 6A.

FIG. 6D shows a diagram illustrating an example of a collectible mediaoverlay 1840. The collectible media overlay 1840 may be randomlysupplied to the mobile device 1802 in response to detecting the mobiledevice 1802 at a geolocation associated with the collectible mediaoverlay 1840. The collectible media overlay 1840 can be stored at themobile device 1802. Once the mobile device 1802 detects that relatedcollectible media overlays have been stored, the mobile device 1802 maycause the related collectible media overlays or a corresponding uniquemedia overlay to be displayed in the display 1804. The mobile device1802 displays the media overlay 1840 on top of the media content 1806 onthe display 1804. In one example embodiment, the media overlay 1840 maybe implemented with the collection module 608 of FIG. 6A.

FIG. 6E shows a diagram illustrating an example of a viral media overlay1850. The viral media overlay 1850 may include dynamic content 1854 andan actionable area 1852. For example, the dynamic content 1854 shows aprogress bar and a goal of a fundraising event. The progress bar isadjusted based on a latest amount raised. The actionable area 1852 maytrigger the mobile device 1802 to cause a financial transaction (e.g.,donation) and a communication to another mobile device (e.g., message toanother mobile device using the messaging application 120). The mobiledevice 1802 displays the viral media overlay 1850 on top of the mediacontent 1806 on the display 1804. In one example embodiment, the viralmedia overlay 1850 may be implemented with the progressive use module610, the viral use module 612, and an actionable module 614 of FIG. 6A.

FIG. 6F shows a diagram illustrating an example of a media overlay 1860with audio content. The media overlay 1860 may include dynamic content1864 that includes, for example, a song title 1862 and a band name. Thedynamic content 1864 may also include an actionable area 1866. Theactionable area 1866 may provide additional information about the song,how to buy the song, etc. The mobile device 1802 may display the mediaoverlay 1860 on top of the media content 1806 (e.g., content item) onthe display 1804. In one example, the media overlay 1860 may beimplemented with the audio module 618 and the actionable module 614 ofFIG. 6A. In the alternative, or in addition, the actionable area 1866may be in another location (e.g., below the media content 1806). Inanother example, a user gesture (e.g., a finger swipe upwards on themedia content 1806), may cause an action to occur (e.g., buy the song,listen to the full track, learn more about the artist, etc.).

FIG. 7 shows an interaction diagram illustrating one example embodimentof an operation of the user-based media overlay publication module 314.At operation 711, the client device 110 of a first user uploads contentand sends a requested geolocation and a requested time duration to themedia overlay application 122. At operation 713, the media overlayapplication 122 generates a media overlay based on the uploaded contentand associates the media overlay with the requested geolocation for therequested time duration. In one example embodiment, operations 711 and713 may be implemented with the user-based media overlay publicationmodule 314 of FIG. 3.

At operation 714, the client device 112 of a second user sendsgeolocation information to the messaging application 120. At operation715, the messaging application 120 identifies, from the media overlayapplication 122, a media overlay based on the geolocation information ofthe client device 112. At operation 718, the media overlay application122 supplies the client device 112 with the identified media overlay. Inone example embodiment, operations 716 and 718 may be implemented withthe media overlay engine 306 of FIG. 3.

FIG. 8 shows an interaction diagram illustrating an example embodimentof an operation of the merchant-based media overlay publication module316. At operation 808, a client device 802 of merchant A uploads contentwith geolocation information (e.g., geolocation X) and a bid amount(e.g., bid amount A) to the media overlay application 122 to form mediaoverlay A. At operation 810, a client device 804 of merchant B uploadscontent with the same geolocation information (e.g., geolocation X) anda bid amount (e.g., bid amount B) to the media overlay application 122to form media overlay B. At operation 812, the media overlay application122 determines a highest bidder, and associates the media overlay of thehighest bidder with geolocation X. For example, if bid amount A isgreater than bid amount B, media overlay A is provided to client devicesthat are located at geolocation X. In one example embodiment, operations808, 810, and 812 may be implemented with the merchant-based mediaoverlay publication module 316 of FIG. 3.

At operation 814, a client device 806 at geolocation X sends itsgeolocation information to the messaging application 120. At operation816, the messaging application 120 identifies, from the media overlayapplication 122, the media overlay associated with the geolocation X. Atoperation 818, the media overlay application 122 supplies the clientdevice 806 with media overlay A. In one example embodiment, operations816 and 818 may be implemented with the media overlay engine 306 of FIG.3. In another example embodiment, the media overlay application 122supplies both media overlays A and B to the client device 806 withinstructions for the client device 806 to display media overlay A firstbefore media overlay B since merchant A was the highest bidder.

FIG. 9 shows a flow diagram illustrating one example embodiment of amethod 901 of the user-based media overlay publication module 314. Atoperation 903, the user-based media overlay publication module 314receives uploaded content and a requested geolocation from a firstclient device. In one example embodiment, operation 903 may beimplemented with the user-based content upload module 402, theuser-based geolocation selection module 404, and the user-based durationselection module 406 of FIG. 4A.

At operation 904, the user-based media overlay publication module 314forms a user-based media overlay that includes the uploaded content andis associated with the requested geolocation. In one example embodiment,operation 904 may be implemented with the user-based publication engine408 of FIG. 4A.

At operation 906, the user-based media overlay publication module 314receives geolocation information from a second client device. Atoperation 908, the user-based media overlay publication module 314determines whether the geolocation of the second client device is withinthe requested geolocation from the first client device. At operation909, the user-based media overlay publication module 314 publishes theuser-based media overlay from the first client device to the secondclient device in response to the geolocation of the second client devicebeing within the requested geolocation from the first client device. Inone example embodiment, operation 909 may be implemented with theuser-based media overlay module 320 of FIG. 3.

At operation 911, the media overlay engine 306 supplies predefined mediaoverlays corresponding to the geolocation of the second client device tothe second client device, in response to the geolocation of the secondclient device not being within the requested geolocation from the firstclient device. In one example embodiment, operation 911 may beimplemented with the predefined media overlay module 318 of FIG. 3.

FIG. 10 shows a flow diagram illustrating one example embodiment of amethod 1000 of operation for the merchant-based media overlaypublication module 316. At operations 1002 and 1004, the merchant-basedmedia overlay publication module 316 receives uploaded content,geolocation information, and corresponding bid amounts from merchants.For example, at operation 1002, the merchant-based content upload module502 receives content A from merchant A. The merchant-based geolocationselection module 504 receives geolocation X from merchant A. Themerchant-based bidding module 508 receives bid amount A from merchant A.

At operation 1004, the merchant-based content upload module 502 receivescontent B from merchant B. The merchant-based geolocation selectionmodule 504 receives geolocation X from merchant B. The merchant-basedbidding module 508 receives bid amount B from merchant B.

At operation 1006, the highest bid amount is determined. In one exampleembodiment, operation 1006 may be implemented with the merchant-basedbidding module 508 of FIG. 6A. If bid amount A is greater than bidamount B, the merchant-based publication engine 510 generates amerchant-based media overlay A based on content A and geolocation X atoperation 1008. At operation 1010, the merchant-based media overlaymodule 322 supplies merchant-based media overlay A to client devicesthat are located at geolocation X.

If bid amount B is greater than bid amount A, the merchant-basedpublication engine 510 generates a merchant-based media overlay B basedon content B and geolocation X at operation 1014. At operation 1016, themerchant-based media overlay module 322 supplies merchant-based mediaoverlay B to client devices that are located at geolocation X.

FIG. 11 shows a flow diagram illustrating one example embodiment of amethod 1100 of operation for the live event module 602. At operation1104, the live event module 602 receives geolocation information from aclient device. At operation 1106, the live event module 602 identifies alive event associated with the geolocation information. At operation1108, the live event module 602 accesses live event data related to thelive event. At operation 1110, the live event module 602 generates alive event media overlay based on the live event data. At operation1112, the live event module 602 supplies the live event media overlay tothe client device.

FIG. 12 shows a flow diagram illustrating one example embodiment of amethod 1200 of operation for the social network module 604. At operation1202, the social network module 604 receives social network informationfrom a client device. At operation 1204, the social network module 604accesses social network data from social network service providers basedon the social network information from the client device. At operation1206, the social network module 604 identifies a geolocation fromgeolocation information of the client device. At operation 1208, thesocial network module 604 generates a social network-based media overlaybased on the social network data and geolocation of the client device.At operation 1210, the social network module 604 supplies the socialnetwork-based media overlay to the client device.

FIG. 13 shows a flow diagram illustrating one example embodiment of amethod 1300 of operation for the promotion module 606. At operation1302, the promotion module 606 generates a set of media overlays for amerchant for a predefined geolocation. At operation 1304, the promotionmodule 606 receives geolocation information from a client device. Atoperation 1306, the promotion module 606 identifies the geolocation ofthe client device from the geolocation information. At operation 1308,the promotion module 606 accesses the set of media overlays for themerchant associated with the geolocation. At operation 1310, thepromotion module 606 randomly selects at least one media overlay fromthe set of media overlays. At operation 1312, the promotion module 606supplies the randomly selected media overlay(s) to the client device.

FIG. 14 shows a flow diagram illustrating one example embodiment of amethod 1400 of operation for the collection module 608. At operation1402, the collection module 608 receives geolocation information from aclient device. At operation 1404, the collection module 608 determinesthe geolocation of the client device from the geolocation information.At operation 1406, the collection module 608 accesses media overlaysassociated with the geolocation of the client device. At operation 1408,the collection module 608 stores the media overlays in a media overlaycollection associated with the client device. At operation 1410, thecollection module 608 presents the media overlays in the media overlaycollection to the client device for use.

FIG. 15 shows a flow diagram illustrating one example embodiment of amethod 1500 of operation for the progressive use module 610. Atoperation 1502, the progressive use module 610 generates a progressiveuse media overlay for a geolocation. At operation 1504, the progressiveuse module 610 receives geolocation information from a first clientdevice at the geolocation. At operation 1506, the progressive use module610 supplies the progressive use media overlay to the first clientdevice, and generates a first modified media overlay based on theprogressive use media overlay. At operation 1508, the progressive usemodule 610 receives geolocation information from a second client deviceat the geolocation. At operation 1510, the progressive use module 610supplies the first modified media overlay to the second client device,and generates a second modified media overlay based on the firstmodified media overlay.

FIG. 16 shows a flow diagram illustrating one example embodiment of amethod 1600 of operation for the viral use module 612. At operation1602, the viral use module 612 generates a media overlay for ageolocation. At operation 1604, the viral use module 612 receives mediametadata from a first client device at the geolocation. At operation1606, the viral use module 612 supplies the media overlay to the firstclient device at the geolocation. At operation 1608, the viral usemodule 612 receives a request from the first client device to forwardthe media overlay to a second client device outside the geolocation. Atoperation 1610, the viral use module 612 provides the media overlay fora limited time to the second client device outside the geolocation.

FIG. 17 shows a flow diagram illustrating one example embodiment of amethod 1700 of operation for the actionable module 614. At operation1702, the actionable module 614 generates an actionable media overlayhaving an actionable portion associated with a function. At operation1704, the actionable module 614 provides the actionable media overlay toa first client device. At operation 1706, the actionable module 614receives a media item (e.g., a photo) with the media overlay from thefirst client device. At operation 1708, the actionable module 614supplies the media item with the media overlay to a second clientdevice. At operation 1710, the actionable module 614 identifies aselection of the actionable portion from the second client device. Atoperation 1712, the actionable module 614 executes a function associatedwith the actionable portion at the second client device.

FIG. 18 shows a flow diagram illustrating one example embodiment of amethod 1800 of operation for a media overlay publication module 304. Atoperation 1882, the media overlay publication module 304 may receive acontent item (e.g., photograph, video, etc.) from a client device 110(for example). For instance, a user of the client device 110 may betaking a photograph or a video using the client device 110. While theuser is taking the photograph or video, or after the user has taken thephotograph or video, the client device 110 may send a portion of thevideo, audio related to the photograph or video (e.g., playing in thebackground), or the captured photograph or video to the media overlaypublication module 304.

At operation 1884, the media overlay publication module 304 maydetermine whether one or more characteristics of the content item matcha trigger for a media overlay. Exemplary characteristics or triggers mayinclude a geolocation associated with the content item or client device110, audio content in the content item, a speed of the client device 110based on movement data from the client device 110, a timestampassociated with the content item, data associated with amachine-readable code (e.g., a bar code, a QR code), an object in thecontent item, adding a new recipient for the content item, etc. If oneor more characteristics of the content item match a trigger for a mediaoverlay, at operation 1886 the media overlay publication module 304 maygenerate a media overlay associated with the content item. A mediaoverlay may include a creative overlay, a geo lens, an audio filter, amusic clip, creative tools, access to additional features orfunctionality, audio clips, etc.

For example, the media overlay publication module 304 may detect audiocontent in the content item and identify an audio track based on theaudio content in the content item (e.g., matching a fingerprint of theaudio content to known fingerprints to identify an audio track). Theaudio track may trigger generation of a media overlay associated withthe audio track. A media overlay may be generated that includes at leastone of a title associated with the audio track, an artist nameassociated with the audio track, an album name associated with the audiotrack, album artwork associated with the audio track, lyrics associatedwith the audio track, etc. (see, e.g., FIG. 6F).

The audio track may be playing at a live event, such as a concert, inone example. Different media overlays may be generated based on whichband is playing at the concert, which song is playing, etc. For example,a media overlay may be generated that includes the particular song andband that is playing, the next song or band that will play, the previoussong or band that played, a highlighted song and/or band amongst a listof songs and/or bands that are playing or will play during the course ofthe event, one or more timestamps for when the song is playing or willend, etc. The media overlay publication module 304 may use otherinformation (e.g., schedule information) associated with the event tosupplement the information received from the client device 110.

In another example, the media overlay may provide additional audiocontent to include with the content item. For example, the content itemmay include one or more photographs or video with a particular audiotrack playing in the background. The media overlay publication module304 may identify the audio track as described above and provide acontinuous version of the audio track set to the photograph(s) or video.

In another example, the media overlay publication module 304 may detecta particular object in the content item (e.g., in a photograph or video)and identify the object (e.g., using object recognition technology). Amedia overlay may be generated associated with the object. For example,a media overlay may be provided to the client device 110 in response tothe user taking a photograph or video of a certain object (e.g., anincentive or reward for taking the photograph or video). The mediaoverlay may be related to the object or have further information aboutthe object. For example, the media overlay publication module 304 mayrecognize an object indicating that the user is at Disneyland andpresent a Disney-themed media overlay. In another example, the mediaoverlay publication module 304 may recognize an object such as a logo,brand name, product, particular person (e.g., celebrity), etc.

In another example, the media overlay publication module 304 may detectdata from a machine-readable code (e.g., barcode, QR code, etc.) in thecontent item. A media overlay may be created or selected based on thecode provided. For example, a machine-readable code may be created for amusical artist and a media overlay may be created for the musicalartist. A user may use the client device 110 to scan themachine-readable code related to the musical artist and the mediaoverlay publication module 304 may generate the media overlay associatedwith the musical artist. Additionally or alternatively, additionalaccessories or features (e.g., tools for editing photographs or video,etc.) may be provided to a user.

In another example, the media overlay publication module 304 may detecta geolocation associated with the content item and generate a mediaoverlay associated with that geolocation (e.g., a media overlay with areal-time score of a game, a media overlay of concert information,etc.), as discussed in further detail above.

At operation 1888, the media overlay publication module 304 may send themedia overlay to the client device 110. The media overlay may bedisplayed on the client device 110 (e.g., overlaid on the photograph orimage), or a notification may be displayed to the user indicating thatone or more media overlays are available for the photograph or video(e.g., audio media overlay, creative tool, etc.) The user may access theone or more media overlays by interacting with the client device 110(e.g., pressing a button, long pressing the screen, swipe or othergesture, etc.). In addition, or alternatively, the media overlay may bealready stored locally on the client device 110 and the media overlaypublication module 304 may send an indication to the client device 110to “unlock” the media overlay to provide to a user.

The user may select to use the media overlay and then send the contentitem including the media overlay to one or more recipients. The mediaoverlay publication module 304 may receive the content item includingthe media overlay in operation 1890. The media overlay publicationmodule 304 may send the content item with the media overlay to a secondclient device 112 (or one or more client devices) at operation 1892. Thecontent item with the media overlay may be displayed on the secondclient device 112.

Before sending the content item with the media overlay to the secondclient device, the media overlay publication module 304 may generate asecond media overlay for the content item. The media overlay publicationmodule 304 may send the content item with the media overlay (e.g., thefirst media overlay) and the second media overlay to the second clientdevice 112. For example, the first media overlay may include audio trackinformation, and the second media overlay may include a link or shortcutto other information about the audio track (e.g., detail about thetrack, artist, album, or lyrics, how to buy the audio track or album,etc.) that can be displayed and/or accessed by a user of the secondclient device. In one example, the second media overlay may be anactionable media overlay as described above.

Characteristics of a content item may trigger other things in additionto or instead of a media overlay. For example, geolocation informationindicating that a user is in a particular store may trigger access to aparticular eCommerce store related to the physical store the user hasentered.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied (1) on a non-transitorymachine-readable medium or (2) in a transmission signal) orhardware-implemented modules. A hardware-implemented module is atangible unit capable of performing certain operations and may beconfigured or arranged in a certain manner. In example embodiments, oneor more computer systems (e.g., a standalone, client, or server computersystem) or one or more processors may be configured by software (e.g.,an application or application portion) as a hardware-implemented modulethat operates to perform certain operations as described herein.

In various embodiments, a hardware-implemented module may be implementedmechanically or electronically. For example, a hardware-implementedmodule may comprise dedicated circuitry or logic that is permanentlyconfigured (e.g., as a special-purpose processor, such as a fieldprogrammable gate array (FPGA) or an application-specific integratedcircuit (ASIC)) to perform certain operations. A hardware-implementedmodule may also comprise programmable logic or circuitry (e.g., asencompassed within a general-purpose processor or other programmableprocessor) that is temporarily configured by software to perform certainoperations. It will be appreciated that the decision to implement ahardware-implemented module mechanically, in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software) may be driven by cost and time considerations.

Accordingly, the term “hardware-implemented module” should be understoodto encompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarily ortransitorily configured (e.g., programmed) to operate in a certainmanner or to perform certain operations described herein. Consideringembodiments in which hardware-implemented modules are temporarilyconfigured (e.g., programmed), each of the hardware-implemented modulesneed not be configured or instantiated at any one instance in time. Forexample, where the hardware-implemented modules comprise ageneral-purpose processor configured using software, the general-purposeprocessor may be configured as respectively differenthardware-implemented modules at different times. Software may,accordingly, configure a processor, for example, to constitute aparticular hardware-implemented module at one instance of time and toconstitute a different hardware-implemented module at a differentinstance of time.

Hardware-implemented modules can provide information to, and receiveinformation from, other hardware-implemented modules. Accordingly, thedescribed hardware-implemented modules may be regarded as beingcommunicatively coupled. Where multiples of such hardware-implementedmodules exist contemporaneously, communications may be achieved throughsignal transmission (e.g., over appropriate circuits and buses thatconnect the hardware-implemented modules). In embodiments in whichmultiple hardware-implemented modules are configured or instantiated atdifferent times, communications between such hardware-implementedmodules may be achieved, for example, through the storage and retrievalof information in memory structures to which the multiplehardware-implemented modules have access. For example, onehardware-implemented module may perform an operation, and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware-implemented module may then,at a later time, access the memory device to retrieve and process thestored output. Hardware-implemented modules may also initiatecommunications with input or output devices, and can operate on aresource (e.g., a collection of information).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods described herein may be at least partiallyprocessor-implemented. For example, at least some of the operations of amethod may be performed by one or more processors orprocessor-implemented modules. The performance of certain of theoperations may be distributed among the one or more processors, not onlyresiding within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment, or a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of theoperations may be performed by a group of computers (as examples ofmachines including processors), with these operations being accessiblevia the network 104 (e.g., the Internet) and via one or more appropriateinterfaces (e.g., APIs).

Example embodiments may be implemented in digital electronic circuitry,or in computer hardware, firmware, or software, or in combinations ofthem. Example embodiments may be implemented using a computer programproduct (e.g., a computer program tangibly embodied in an informationcarrier, e.g., in a machine-readable medium for execution by, or tocontrol the operation of, data processing apparatus, e.g., aprogrammable processor, a computer, or multiple computers).

A computer program can be written in any form of programming language,including compiled or interpreted languages, and it can be deployed inany form, including as a standalone program or as a module, subroutine,or other unit suitable for use in a computing environment. A computerprogram can be deployed to be executed on one computer or on multiplecomputers at one site or distributed across multiple sites andinterconnected by a communication network.

In example embodiments, operations may be performed by one or moreprogrammable processors executing a computer program to performfunctions by operating on input data and generating output. Methodoperations can also be performed by, and apparatus of exampleembodiments may be implemented as, special purpose logic circuitry(e.g., an FPGA or an ASIC).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. Inembodiments deploying a programmable computing system, it will beappreciated that both hardware and software architectures meritconsideration. Specifically, it will be appreciated that the choice ofwhether to implement certain functionality in permanently configuredhardware (e.g., an ASIC), in temporarily configured hardware (e.g., acombination of software and a programmable processor), or in acombination of permanently and temporarily configured hardware may be adesign choice. Below are set out hardware (e.g., machine) and softwarearchitectures that may be deployed in various example embodiments.

FIG. 19 is a block diagram 700 illustrating an architecture of software702, which can be installed on any one or more of the devices describedabove. For example, in various embodiments, the client devices 110 and112 and the data exchange platform 102 (e.g., application server 118,database server 124, etc.) may be implemented using some or all of theelements of the software 702. FIG. 19 is merely a non-limiting exampleof a software architecture, and it will be appreciated that many otherarchitectures can be implemented to facilitate the functionalitydescribed herein. In various embodiments, the software 702 isimplemented by hardware such as a machine 900 of FIG. 20 that includesprocessors 910, memory 930, and I/O components 950. In this examplearchitecture, the software 702 can be conceptualized as a stack oflayers where each layer may provide a particular functionality. Forexample, the software 702 includes layers such as an operating system704, libraries 706, frameworks 708, and applications 710. Operationally,the applications 710 invoke API calls 712 through the software stack andreceive messages 714 in response to the API calls 712, consistent withsome embodiments.

In various implementations, the operating system 704 manages hardwareresources and provides common services. The operating system 704includes, for example, a kernel 720, services 722, and drivers 724. Thekernel 720 acts as an abstraction layer between the hardware and theother software layers, consistent with some embodiments. For example,the kernel 720 provides memory management, processor management (e.g.,scheduling), component management, networking, and security settings,among other functionality. The services 722 can provide other commonservices for the other software layers. The drivers 724 are responsiblefor controlling or interfacing with the underlying hardware, accordingto some embodiments. For instance, the drivers 724 can include displaydrivers, camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers,flash memory drivers, serial communication drivers (e.g., UniversalSerial Bus (USB) drivers), WI-FI® drivers, audio drivers, powermanagement drivers, and so forth.

In some embodiments, the libraries 706 provide a low-level commoninfrastructure utilized by the applications 710. The libraries 706 caninclude system libraries 730 (e.g., C standard library) that can providefunctions such as memory allocation functions, string manipulationfunctions, mathematic functions, and the like. In addition, thelibraries 706 can include API libraries 732 such as media libraries(e.g., libraries to support presentation and manipulation of variousmedia formats such as Moving Picture Experts Group-4 (MPEG4), AdvancedVideo Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3),Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec,Joint Photographic Experts Group (JPEG or JPG), or Portable NetworkGraphics (PNG)), graphics libraries (e.g., an OpenGL framework used torender in two dimensions (2D) and three dimensions (3D) in graphiccontent on a display), database libraries (e.g., SQLite to providevarious relational database functions), web libraries (e.g., WebKit toprovide web browsing functionality), and the like. The libraries 706 canalso include a wide variety of other libraries 734 to provide many otherAPIs to the applications 710.

The frameworks 708 provide a high-level common infrastructure that canbe utilized by the applications 710, according to some embodiments. Forexample, the frameworks 708 provide various graphic user interface (GUI)functions, high-level resource management, high-level location services,and so forth. The frameworks 708 can provide a broad spectrum of otherAPIs that can be utilized by the applications 710, some of which may bespecific to a particular operating system 704 or platform.

In an example embodiment, the applications 710 include a homeapplication 750, a contacts application 752, a browser application 754,a book reader application 756, a location application 758, a mediaapplication 760, a messaging application 762, a game application 764,and a broad assortment of other applications such as a third partyapplication 766. According to some embodiments, the applications 710 areprograms that execute functions defined in the programs. Variousprogramming languages can be employed to create one or more of theapplications 710, structured in a variety of manners, such asobject-oriented programming languages (e.g., Objective-C, Java, or C++)or procedural programming languages (e.g., C or assembly language). In aspecific example, the third party application 766 (e.g., an applicationdeveloped using the ANDROID™ or IOS™ software development kit (SDK) byan entity other than the vendor of the particular platform) may bemobile software running on a mobile operating system such as IOS™,ANDROID™, WINDOWS® Phone, or another mobile operating system. In thisexample, the third party application 766 can invoke the API calls 712provided by the operating system 704 to facilitate functionalitydescribed herein.

Some embodiments may particularly include a content gallery application767. In certain embodiments, this may be a standalone application thatoperates to manage communications with a server system such as the dataexchange platform 102. In other embodiments, this functionality may beintegrated with another application, such as the media application 760or another such application. The content gallery application 767 maymanage collection of content using a camera device of the machine 900,communication with a server system via the I/O components 950, andreceipt and storage of received content in the memory 930. Presentationof content and user inputs associated with content may be managed by thecontent gallery application 767 using different frameworks 708, library706 elements, or operating system 704 elements operating on the machine900.

FIG. 20 is a block diagram illustrating components of a machine 900,according to some embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.Specifically, FIG. 20 shows a diagrammatic representation of the machine900 in the example form of a computer system, within which instructions916 (e.g., software, a program, an application 710, an applet, an app,or other executable code) for causing the machine 900 to perform any oneor more of the methodologies discussed herein can be executed. Inalternative embodiments, the machine 900 operates as a standalone deviceor can be coupled (e.g., networked) to other machines. In a networkeddeployment, the machine 900 may operate in the capacity of a servermachine (e.g., application server 118, database server 124) or a clientdevice 110, 112 in a server-client network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine 900 can comprise, but not be limited to, a server computer, aclient computer, a personal computer (PC), a tablet computer, a laptopcomputer, a netbook, a set-top box (STB), a personal digital assistant(PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smarthome device (e.g., a smart appliance), other smart devices, a webappliance, a network router, a network switch, a network bridge, or anymachine capable of executing the instructions 916, sequentially orotherwise, that specify actions to be taken by the machine 900. Further,while only a single machine 900 is illustrated, the term “machine” shallalso be taken to include a collection of machines 900 that individuallyor jointly execute the instructions 916 to perform any one or more ofthe methodologies discussed herein.

In various embodiments, the machine 900 comprises processors 910, memory930, and I/O components 950, which can be configured to communicate witheach other via a bus 902. In an example embodiment, the processors 910(e.g., a central processing unit (CPU), a reduced instruction setcomputing (RISC) processor, a complex instruction set computing (CISC)processor, a graphics processing unit (GPU), a digital signal processor(DSP), an application specific integrated circuit (ASIC), aradio-frequency integrated circuit (RFIC), another processor, or anysuitable combination thereof) include, for example, a processor 912 anda processor 914 that may execute the instructions 916. The term“processor” is intended to include multi-core processors 910 that maycomprise two or more independent processors 912, 914 (also referred toas “cores”) that can execute instructions 916 contemporaneously.Although FIG. 19 shows multiple processors 910, the machine 900 mayinclude a single processor 910 with a single core, a single processor910 with multiple cores (e.g., a multi-core processor 910), multipleprocessors 912, 914 with a single core, multiple processors 912, 914with multiples cores, or any combination thereof.

The memory 930 comprises a main memory 932, a static memory 934, and astorage unit 936 accessible to the processors 910 via the bus 902,according to some embodiments. The storage unit 936 can include amachine-readable medium 938 on which are stored the instructions 916embodying any one or more of the methodologies or functions describedherein. The instructions 916 can also reside, completely or at leastpartially, within the main memory 932, within the static memory 934,within at least one of the processors 910 (e.g., within the processor'scache memory), or any suitable combination thereof, during executionthereof by the machine 900. Accordingly, in various embodiments, themain memory 932, the static memory 934, and the processors 910 areconsidered machine-readable media 938.

As used herein, the term “memory” refers to a machine-readable medium938 able to store data temporarily or permanently and may be taken toinclude, but not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, and cache memory. While themachine-readable medium 938 is shown, in an example embodiment, to be asingle medium, the term “machine-readable medium” should be taken toinclude a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storethe instructions 916. The term “machine-readable medium” shall also betaken to include any medium, or combination of multiple media, that iscapable of storing instructions (e.g., instructions 916) for executionby a machine (e.g., machine 900), such that the instructions 916, whenexecuted by one or more processors of the machine 900 (e.g., processors910), cause the machine 900 to perform any one or more of themethodologies described herein. Accordingly, a “machine-readable medium”refers to a single storage apparatus or device, as well as “cloud-based”storage systems or storage networks that include multiple storageapparatus or devices. The term “machine-readable medium” shallaccordingly be taken to include, but not be limited to, one or more datarepositories in the form of a solid-state memory (e.g., flash memory),an optical medium, a magnetic medium, other non-volatile memory (e.g.,erasable programmable read-only memory (EPROM)), or any suitablecombination thereof. The term “machine-readable medium” specificallyexcludes non-statutory signals per se.

The I/O components 950 include a wide variety of components to receiveinput, provide output, produce output, transmit information, exchangeinformation, capture measurements, and so on. In general, it will beappreciated that the I/O components 950 can include many othercomponents that are not shown in FIG. 20. The I/O components 950 aregrouped according to functionality merely for simplifying the followingdiscussion, and the grouping is in no way limiting. In various exampleembodiments, the I/O components 950 include output components 952 andinput components 954. The output components 952 include visualcomponents (e.g., a display such as a plasma display panel (PDP), alight emitting diode (LED) display, a liquid crystal display (LCD), aprojector, or a cathode ray tube (CRT)), acoustic components (e.g.,speakers), haptic components (e.g., a vibratory motor), other signalgenerators, and so forth. The input components 954 include alphanumericinput components (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point-based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstruments), tactile input components (e.g., a physical button, a touchscreen that provides location and force of touches or touch gestures, orother tactile input components), audio input components (e.g., amicrophone), and the like.

In some further example embodiments, the I/O components 950 includebiometric components 956, motion components 958, environmentalcomponents 960, or position components 962, among a wide array of othercomponents. For example, the biometric components 956 include componentsto detect expressions (e.g., hand expressions, facial expressions, vocalexpressions, body gestures, or eye tracking), measure biosignals (e.g.,blood pressure, heart rate, body temperature, perspiration, or brainwaves), identify a person (e.g., voice identification, retinalidentification, facial identification, fingerprint identification, orelectroencephalogram based identification), and the like. The motioncomponents 958 include acceleration sensor components (e.g.,accelerometer), gravitation sensor components, rotation sensorcomponents (e.g., gyroscope), and so forth. The environmental components960 include, for example, illumination sensor components (e.g.,photometer), temperature sensor components (e.g., one or morethermometers that detect ambient temperature), humidity sensorcomponents, pressure sensor components (e.g., barometer), acousticsensor components (e.g., one or more microphones that detect backgroundnoise), proximity sensor components (e.g., infrared sensors that detectnearby objects), gas sensor components (e.g., machine olfactiondetection sensors, gas detection sensors to detect concentrations ofhazardous gases for safety or to measure pollutants in the atmosphere),or other components that may provide indications, measurements, orsignals corresponding to a surrounding physical environment. Theposition components 962 include location sensor components (e.g., a GPSreceiver component), altitude sensor components (e.g., altimeters orbarometers that detect air pressure from which altitude may be derived),orientation sensor components (e.g., magnetometers), and the like.

Communication can be implemented using a wide variety of technologies.The I/O components 950 may include communication components 964 operableto couple the machine 900 to a network 980 or devices 970 via a coupling982 and a coupling 972, respectively. For example, the communicationcomponents 964 include a network interface component or another suitabledevice to interface with the network 980. In further examples, thecommunication components 964 include wired communication components,wireless communication components, cellular communication components,near field communication (NFC) components, BLUETOOTH® components (e.g.,BLUETOOTH® Low Energy), WI-FI® components, and other communicationcomponents to provide communication via other modalities. The devices970 may be another machine 900 or any of a wide variety of peripheraldevices (e.g., a peripheral device coupled via a Universal Serial Bus(USB)).

Moreover, in some embodiments, the communication components 964 detectidentifiers or include components operable to detect identifiers. Forexample, the communication components 964 include radio frequencyidentification (RFID) tag reader components, NFC smart tag detectioncomponents, optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as a Universal Product Code (UPC) barcode, multi-dimensional bar codes such as a Quick Response (QR) code,Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code,Uniform Commercial Code Reduced Space Symbology (UCC RSS)-2D bar codes,and other optical codes), acoustic detection components (e.g.,microphones to identify tagged audio signals), or any suitablecombination thereof. In addition, a variety of information can bederived via the communication components 964, such as location viaInternet Protocol (IP) geolocation, location via WI-FI® signaltriangulation, location via detecting a BLUETOOTH® or NFC beacon signalthat may indicate a particular location, and so forth.

In various example embodiments, one or more portions of the network 980can be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the publicswitched telephone network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a WI-FI®network, another type of network, or a combination of two or more suchnetworks. For example, the network 980 or a portion of the network 980may include a wireless or cellular network, and the coupling 982 may bea Code Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or another type of cellular orwireless coupling. In this example, the coupling 982 can implement anyof a variety of types of data transfer technology, such as SingleCarrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized(EVDO) technology, General Packet Radio Service (GPRS) technology,Enhanced Data rates for GSM Evolution (EDGE) technology, thirdGeneration Partnership Project (3GPP) including 3G, fourth generationwireless (4G) networks, Universal Mobile Telecommunications System(UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability forMicrowave Access (WiMAX), Long Term Evolution (LTE) standard, othersdefined by various standard-setting organizations, other long rangeprotocols, or other data transfer technology.

In example embodiments, the instructions 916 are transmitted or receivedover the network 980 using a transmission medium via a network interfacedevice (e.g., a network interface component included in thecommunication components 964) and utilizing any one of a number ofwell-known transfer protocols (e.g., Hypertext Transfer Protocol(HTTP)). Similarly, in other example embodiments, the instructions 916are transmitted or received using a transmission medium via the coupling972 (e.g., a peer-to-peer coupling) to the devices 970. The term“transmission medium” shall be taken to include any intangible mediumthat is capable of storing, encoding, or carrying the instructions 916for execution by the machine 900, and includes digital or analogcommunications signals or other intangible media to facilitatecommunication of such software.

Furthermore, the machine-readable medium 938 is non-transitory (in otherwords, not having any transitory signals) in that it does not embody apropagating signal. However, labeling the machine-readable medium 938“non-transitory” should not be construed to mean that the medium isincapable of movement; the medium 938 should be considered as beingtransportable from one physical location to another. Additionally, sincethe machine-readable medium 938 is tangible, the medium 938 may beconsidered to be a machine-readable device.

FIG. 21 is a block diagram illustrating an exemplary mobile device 1900,according to an example embodiment. The mobile device 1900 may include aprocessor 1902. The processor 1902 may be any of a variety of differenttypes of commercially available processors 1902 suitable for mobiledevices 1900 (for example, an XScale architecture microprocessor, amicroprocessor without interlocked pipeline stages (MIPS) architectureprocessor, or another type of processor 1902). A memory 1904, such as arandom access memory (RAM), a flash memory, or another type of memory,is typically accessible to the processor 1902. The memory 1904 may beadapted to store an operating system (OS) 1906, as well as applications1908, such as a mobile location enabled application that may providelocation-based services (LBSs) to a user. The processor 1902 may becoupled, either directly or via appropriate intermediary hardware, to adisplay 1910 and to one or more input/output (I/O) devices 1912, such asa keypad, a touch panel sensor, a microphone, and the like. Similarly,in some embodiments, the processor 1902 may be coupled to a transceiver1914 that interfaces with an antenna 1916. The transceiver 1914 may beconfigured to both transmit and receive cellular network signals,wireless data signals, or other types of signals via the antenna 1916,depending on the nature of the mobile device 1900. Further, in someconfigurations, a GPS receiver 1918 may also make use of the antenna1916 to receive GPS signals.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the inventive subject matter may be referred to herein, individuallyor collectively, by the term “invention” merely for convenience andwithout intending to voluntarily limit the scope of this application toany single disclosure or inventive concept if more than one is, in fact,disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within a scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b), requiring an abstract that will allow the reader to quicklyascertain the nature of the technical disclosure. It is submitted withthe understanding that it will not be used to interpret or limit thescope or meaning of the claims. In addition, in the foregoing DetailedDescription, it can be seen that various features are grouped togetherin a single embodiment for the purpose of streamlining the disclosure.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus, the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate embodiment.

What is claimed is:
 1. A method comprising: receiving from a firstclient device, at a server computer associated with a media overlaypublication system, content to generate a media overlay for augmentingan image or video based on context data corresponding to the image orvideo; receiving, from the first client device, a selected geolocationindicating a trigger upon which the media overlay is provided, theselected geolocation specifying boundaries for the selected geolocation;generating, by the server computer, the media overlay using the contentreceived from the first client device; storing, by the server computer,the generated media overlay in a database associated with the servercomputer; associating, by the server computer, the stored media overlaywith the selected geolocation received from the first client device as atrigger for providing the media overlay; receiving, by the servercomputer, location data from a plurality of client devices; determining,based on the location data, that a location of each of a subset of theplurality of client devices is within the boundaries for the selectedgeolocation triggering the media overlay; and publishing the mediaoverlay to each of the subset of client devices having a location withinthe boundaries of the selected geolocation, wherein the media overlay isdisplayed on a user interface of each of the subset of client devicesfor selection to augment a respective image or video captured by each ofthe client devices.
 2. The method of claim 1, wherein the contentcomprises a logo or other indicia related to a merchant.
 3. The methodof claim 1, wherein the selected geolocation is based on a selection ofa geolocation on a map displayed on the first client device.
 4. Themethod of claim 1, further comprising: determining which client devicesof the subset of the plurality of client devices is associated with acontact of a user of the first computing device; and only publishing themedia overlay to the client devices of the subset of the plurality ofclient devices that are associated with a contact of a user of the firstcomputing device.
 5. The method of claim 1, wherein the media overlay isapplied to an image or video on a user interface of at least one clientdevice of the subset of client devices.
 6. The method of claim 1,further comprising: receiving a bid amount for the media overlay; andassociating the bid amount with the media overlay.
 7. The method ofclaim 1, wherein the content comprises an image or video.
 8. The methodof claim 1, further comprising: receiving an existing establishment orpoint of interest indicating a second trigger upon which the mediaoverlay is provided; and publishing the media overlay to each of asecond subset of the plurality of client devices having a location nearthe point of interest or near the existing establishment.
 9. The methodof claim 1, further comprising: receiving time duration informationindicating how long a media overlay is associated with the selectedgeolocation, wherein the time duration information is for a specifictime period or the time duration information is for a reoccurring timeperiod; and associating the time duration information with the mediaoverlay.
 10. The method of claim 9, wherein the media overlay is nolonger accessible at the selected geolocation after the specified timeperiod or the reoccurring time period has elapsed.
 11. A server computercomprising: one or more processors; and a non-transitorycomputer-readable storage medium comprising instructions that, whenexecuted by the computing device, cause the server computer to performoperations comprising: receiving from a first client device, content togenerate a media overlay for augmenting an image or video based oncontext data corresponding to the image or video; receiving, from thefirst client device, a selected geolocation indicating a trigger uponwhich the media overlay is provided, the selected geolocation specifyingboundaries for the selected geolocation; generating the media overlayusing the content received from the first client device; storing thegenerated media overlay in a database associated with the servercomputer; associating the stored media overlay with the selectedgeolocation received from the first client device as a trigger forproviding the media overlay; receiving location data from a plurality ofclient devices; determining, based on the location data, that a locationof each of a subset of the plurality of client devices is within theboundaries for the selected geolocation triggering the media overlay;and publishing the media overlay to each of the subset of client deviceshaving a location within the boundaries of the selected geolocation,wherein the media overlay is displayed on a user interface of each ofthe subset of client devices for selection to augment a respective imageor video captured by each of the client devices.
 12. The server computerof claim 11, wherein the content comprises a logo or other indiciarelated to a merchant.
 13. The server computer of claim 11, wherein theselected geolocation is based on a selection of a geolocation on a mapdisplayed on the first client device.
 14. The server computer of claim11, the operations further comprising: determining which client devicesof the subset of the plurality of client devices is associated with acontact of a user of the first computing device; and only publishing themedia overlay to the client devices of the subset of the plurality ofclient devices that are associated with a contact of a user of the firstcomputing device.
 15. The server computer of claim 11, wherein the mediaoverlay is applied to an image or video on a user interface of at leastone client device of the subset of client devices.
 16. The servercomputer of claim 11, the operations further comprising: receiving a bidamount for the media overlay; and associating the bid amount with themedia overlay.
 17. The server computer of claim 11, wherein the contentcomprises an image or video.
 18. The server computer of claim 11, theoperations further comprising: receiving an existing establishment orpoint of interest indicating a second trigger upon which the mediaoverlay is provided; and publishing the media overlay to each of asecond subset of the plurality of client devices having a location nearthe point of interest or near the existing establishment.
 19. The servercomputer of claim 11, the operations further comprising: receiving timeduration information indicating how long a media overlay is associatedwith the selected geolocation, wherein the time duration information isfor a specific time period or the time duration information is for areoccurring time period; and associating the time duration informationwith the media overlay.
 20. A non-transitory computer-readable storagemedium comprising instructions that, when executed by one or moreprocessors of a computing device, cause the computing device to performoperations comprising: receiving from a first client device, content togenerate a media overlay for augmenting an image or video based oncontext data corresponding to the image or video; receiving, from thefirst client device, a selected geolocation indicating a trigger uponwhich the media overlay is provided, the selected geolocation specifyingboundaries for the selected geolocation; generating the media overlayusing the content received from the first client device; storing thegenerated media overlay in a database associated with the servercomputer; associating the stored media overlay with the selectedgeolocation received from the first client device as a trigger forproviding the media overlay; receiving location data from a plurality ofclient devices; determining, based on the location data, that a locationof each of a subset of the plurality of client devices is within theboundaries for the selected geolocation triggering the media overlay;and publishing the media overlay to each of the subset of client deviceshaving a location within the boundaries of the selected geolocation,wherein the media overlay is displayed on a user interface of each ofthe subset of client devices for selection to augment a respective imageor video captured by each of the client devices.